Generating Canonical Example Sentences using Candidate Words

Situations where a spoken dialogue system cannot interpret a user’s utterance are a major source of frustration in humancomputer spoken dialogue. Current spoken dialogue systems generally respond with an unhelpful “I’m sorry, I didn’t understand,” or something similarly uninformative. Recent work on Targeted Help has shown that giving users more appropriate feedback makes systems easier to learn and improves performance. In particular, Targeted Help can give the user an appropriate within-domain example sentence to help the user more quickly learn the system’s lexical and grammatical coverage. This paper addresses the generation problem of how to find such an example sentence. We present and evaluate four algorithms for solving this generation problem: an Iterative-Deepening (ID) algorithm, an algorithm, a combined -ID algorithm, and an anytime algorithm. Introduction The goal of Targeted Help in a spoken dialogue system is to provide the user with more specific information in cases in which the system is not able to understand their speech. For instance, if the user is unfamiliar with a push-to-talk device, the system may recognize that the user was already speaking when the push-to-talk button was pressed, and the system might respond with Please be sure that the button is pressed before you begin speaking. Previous work comparing grammar-based language models with statistical language models (Knight et al. 2001) has shown that grammarbased language models show superior recognition accuracy for within-grammar utterances compared to a statistical language model. Systems that use grammar-based language models will then benefit from teaching the user the bounds of the system quickly, and encouraging them to use withingrammar utterances. Recent work (Hockey et al. 2003) has shown that Targeted Help leads to increased task-success and decreased time-to-complete on a urban patrol and search and rescue task in a mixed-initiative dialogue system to control a simulated robotic helicopter. There are a number of strategies for what type of Targeted Help system response might be produced. The previous example illustrates a diagnostic response. Another possibilThe research reported in this paper performed at RIACS was supported under NASA Cooperative Agreement Number NCC 21006. ity is to produce an in-coverage example sentence similar to what the user said. Some obvious questions are: 1) what can be known about the user’s utterance if the system is not able to understand it? and 2) in what way can the generated utterance be made similar to the user’s utterance? Even when the user produces an out-of-coverage utterance they are likely to produce some in-coverage words. Our Targeted Help system runs a fall-back speech recognizer that is being driven by a category-based statistical language model. When the grammar-based recognizer fails, the system looks for within-domain words in the recognition hypothesis from the fall-back recognizer. This gives us a set of target words (potentially with word-confidence scores), and we then try to generate a grammatical example utterance containing those words. For example, if the user says something like I’d like the pressure at the commander, and the fall-back recognizer detected the words pressure and commander, the generator could provide an example grammatical utterance like measure the pressure at the commander’s seat. This paper focusses on algorithms for generating this type of in-coverage example. The choice of an ideal example sentence could conceiveably take into account information from a wide variety of sources, including discourse history, user model, and pedagogical strategy. This could lead to constraints on not only what words should be included in the example, but also what syntactic structures, semantic representations, and word order should be used. For this paper, we have settled on an initial simplification of this task, using only a set of desirable candidate words. Our definition of this generation task is then: Given a grammar G, and a set of target words , find a word string such that